Upgrade llama.cpp from b9071 to b9094#115
Merged
Merged
Conversation
No breaking changes to JNI layer. New upstream features: 2-GPU PCIe AllReduce (CUDA), Snake activation fusion (CUDA), flash attention head size 192, Q5_K SYCL reorder, Hexagon GATED_DELTA_NET/L2_NORM, Sarvam-MoE model, Gemma4 split experts, n_ctx in /v1/models, GCP/Vertex AI compat server endpoint, router child→parent model info propagation. https://claude.ai/code/session_01Jbb6GBWWnZ94gsdFeuKHNw
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. | ||
|
|
||
| Current llama.cpp pinned version: **b9071** | ||
| Current llama.cpp pinned version: **b9094** |
There was a problem hiding this comment.
✓ Correctly updated version from b9071 to b9094 as required by the upgrade procedure (also updated in CMakeLists.txt and README.md).
Review Summary✅ Version Upgrade Procedure: Correctly updated all three required files:
✅ Changelog Documentation: The added 12 rows (b9071–b9094) are comprehensive and well-structured:
✅ Key Safety Notes:
No blockers detected. The upgrade follows documented procedures and introduces no visible breaking changes to the Java API. |
Comment on lines
+231
to
+242
| | ~b9071–b9094 | `ggml/src/ggml-cuda/allreduce.cu` + `allreduce.cuh` (NEW) | 2-GPU PCIe AllReduce pipeline for tensor parallelism (no NVLink required); requires Volta+ (sm70+); enabled via `GGML_CUDA_ALLREDUCE` env var (`nccl`/`internal`/`none`); compiled automatically via FetchContent, no project changes required | | ||
| | ~b9071–b9094 | `ggml/src/ggml-cuda/snake.cu` + `snake.cuh` (NEW) | Fused CUDA Snake activation kernel (`y = x + sin(a*x)^2 * inv_b`) for BigVGAN/Vocos audio models; fuses 5-op chain `MUL→SIN→SQR→MUL→ADD` at graph level; F32/F16/BF16; compiled automatically, no project changes required | | ||
| | ~b9071–b9094 | `ggml/src/ggml-cuda/ggml-cuda.cu` | Flash attention head size 192 (DKQ=192, DV=128) for MiMo-V2.5/V2.5-Pro/V2-Flash with GQA ratio 8/16; multi-GPU comm context refactored to `ggml_backend_cuda_comm_context` with `try_allreduce` function pointer; PCI bus IDs lowercased; compiled automatically, no project changes required | | ||
| | ~b9071–b9094 | `ggml/src/ggml-sycl/` | Q5_K reordered memory layout + MMVQ kernel for Intel GPUs; PAD op supports non-contiguous src0; dedicated growing K/V buffer for flash attention; all internal SYCL backend, no project changes required | | ||
| | ~b9071–b9094 | `ggml/src/ggml-hexagon/` | GATED_DELTA_NET and L2_NORM HVX-vectorized on Hexagon HTP backend; internal DSP backend, no project changes required | | ||
| | ~b9071–b9094 | `src/models/sarvam.cpp` (NEW) | Sarvam-MoE model (`sarvamai/sarvam-30b`); reuses BailingMoeV2 arch; new vocab pre-type `LLAMA_VOCAB_PRE_TYPE_SARVAM_MOE = 51`; additive, no project changes required | | ||
| | ~b9071–b9094 | `src/models/gemma4.cpp` | Gemma4 split gate/up experts: `ffn_gate_up_exps` now TENSOR_NOT_REQUIRED; fallback to separate `ffn_gate_exps`/`ffn_up_exps`; NVFP4 per_expert_scale folding; internal model-loading, no project changes required | | ||
| | ~b9071–b9094 | `tools/server/server-context.h` + `server-context.cpp` | New `get_model_info()` method on `server_context`; `/v1/models` response now includes `"n_ctx"` field (value: `slot_n_ctx`); compiled from upstream sources, no JNI changes required (Java callers of model info APIs receive the new field transparently) | | ||
| | ~b9071–b9094 | `tools/server/server-http.h` + `server.cpp` | `handlers` map moved from private to public in `server_http_context`; new `register_gcp_compat()` method exposes GCP/Vertex AI Prediction Protocol endpoint reading `AIP_MODE`/`AIP_PREDICT_ROUTE`/`AIP_HEALTH_ROUTE`/`AIP_HTTP_PORT` env vars; compiled from upstream sources, no project changes required | | ||
| | ~b9071–b9094 | `tools/server/server-models.h` + `server.cpp` | Router child→parent model info propagation: new `CMD_CHILD_TO_ROUTER_INFO` command; `setup_child_server()` gains `const json & model_info` parameter; new `update_loaded_info()` method; `server_model_meta` gains `loaded_info` field; all internally consistent across compiled upstream sources, no project changes required | | ||
| | ~b9071–b9094 | `common/reasoning-budget.cpp` | Forced token logit no longer set to `+INFINITY`; only competing tokens set to `-INFINITY`; internal sampler behavior change, no project changes required | | ||
| | ~b9071–b9094 | `tools/server/webui/` | Settings registry refactored (`settings-config.ts`/`settings-fields.ts`/`settings-sections.ts` merged into `settings-registry.ts`); MCP route `#/settings/mcp` → `#/mcp-servers`; settings route `/settings/chat/[section]` → `/settings/[[section]]`; UI-only, no project changes required | |
There was a problem hiding this comment.
Excellent changelog entries (b9071–b9094). Each row clearly documents:
- What changed and where (file paths)
- The technical detail (kernel improvements, new models, API additions)
- Project impact (all correctly marked as "no project changes required" or "transparent to Java callers")
Particularly good: The model info API note explicitly confirms that Java callers transparently receive new fields, and the reasoning budget behavior change is correctly identified as internal sampler-only with zero JNI impact.
| @@ -1,5 +1,5 @@ | |||
|  | |||
| [](https://github.com/ggml-org/llama.cpp/releases/tag/b9071) | |||
| [](https://github.com/ggml-org/llama.cpp/releases/tag/b9094) | |||
There was a problem hiding this comment.
✓ Badge and link correctly updated from b9071 to b9094, keeping the badge href in sync with the displayed version.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR upgrades the pinned llama.cpp version from b9071 to b9094, incorporating upstream improvements across CUDA kernels, model support, server functionality, and UI components.
Key Changes
CUDA & Backend Enhancements
GGML_CUDA_ALLREDUCEenv vary = x + sin(a*x)^2 * inv_boperationggml_backend_cuda_comm_contextwithtry_allreducefunction pointerModel Support
sarvamai/sarvam-30b) with new vocab pre-typeLLAMA_VOCAB_PRE_TYPE_SARVAM_MOE = 51Server Improvements
get_model_info()method;/v1/modelsendpoint now includesn_ctxfieldregister_gcp_compat()method supporting AIP environment variablesCMD_CHILD_TO_ROUTER_INFOcommand-INFINITY)UI Updates
#/settings/mcp→#/mcp-servers); settings route generalizedNotes
All changes are compiled from upstream sources. No JNI layer modifications are required; Java callers transparently receive new fields and functionality.
https://claude.ai/code/session_01Jbb6GBWWnZ94gsdFeuKHNw